Towards self-organising action selection

Systems with multiple parallel goals (e.g. autonomous mobile robots) have a problem analogous to that of action selection in ethology. Archi-tectures such as the subsumption architecture (Brooks) involve multiple sensing-and-acting agents within a single robot, more than one of which is capable of controlling the robot on its own if allowed. Which to give control to at a given moment is normally regarded as a (diicult) problem of design. In a quest for a scheme where the agents decide for themselves in a sensible manner, I introduce a model where the agents are not only autonomous but are in full competition with each other for control of the robot. Interesting robots are ones where no agent achieves total victory, but rather a series of compromises are reached. Having the agents operate by the reinforcement learning algorithm Q-learning (Watkins) allows the introduction of an algorithm called`W-learning', by which the agents learn to focus their competitive eeorts in a manner similar to agents with limited spending power in an economy. In this way, the population of agents organises its own action selection in a coherent way that supports parallelism and opportunism. In the empirical section, I show how the relative innuence an agent has on its robot may be controlled by adjusting its rewards. The possibility of automated search of agent-combinations is considered. 1 Action selection It is generally accepted in ethology that animals have multiple competing desires or goals, among which attention must endlessly switch. The animal must solve the problem of action selection-the choosing of the appropriate goal to pursue or desire to satisfy at a given moment in time, given the state of the external world. In recent years, roboticists (and other AI workers) have become interested in models of such systems with multiple parallel goals.